Animation of F1 Score

How performance metric contours change with class balance

---

---

These animated plots show how the contours of F1 Score (F1) change with class balance, i.e., as the number of negative examples (\(n\)) and positive examples (\(p\)) vary in confusion matrices of fixed size (\(N\)).

Each frame of this animation shows a two dimensional slice through the tetrahedral confusion simplex, a projection of the four dimensional confusion matrices of size \(100\) into three dimensions. The animation shows slices sweeping from the edge of the simplex where \(\mathrm{TP}=p, \mathrm{TN}=n\) through to the edge where \(\mathrm{FN}=p, \mathrm{FP}=n\).

Each coloured point corresponds to a specific confusion matrix in which \[ \begin{bmatrix} \mathrm{TP} & \mathrm{FP}\\ \mathrm{FN} & \mathrm{TN} \end{bmatrix}= \begin{bmatrix} \mathrm{TP} & \mathrm{FP}\\ p-\mathrm{TP} & n-\mathrm{FP} \end{bmatrix} \] and \(N=p+n=100\). Hence, for a given \(p\) and \(n\), we can plot the \((p+1)\times(n+1)\) points whose \(\mathrm{TP}\) values range from \(0\) to \(p\) and whose \(\mathrm{FP}\) values range from \(0\) to \(n\) while overlaying the contours of the F1 Score performance metric ranging from \(-0.9, -0.8, \dots, 0.9\).

Note that

• The contours of the F1 Score performance metric are defined continuously, but empirical confusion matrices can only take on values at the discrete points in these plots.
• The left hand plot shows these points and F1 Score contours in ROC space in which a classifier’s true positive is plotted against its false positive rate in the space of rational numbers from \([0,1]\times[0,1]\).
• The right hand plot shows these points and F1 Score contours as an orthographic projection of the slice of points from the confusion simplex.
• The left hand ROC plot is is equivalent to re-scaling the \(x\)-axis of (a) by a factor of \(\tfrac1n\) and the \(y\)-axis by \(\tfrac1p\).
• As we show elsewhere, the F1 Score contours describe a series of lines which intersect at \((\mathrm{FPR}, \mathrm{TPR}) = \left(0,-\tfrac{p}{n}\right)\).